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FOREWORD 



* The Army Research Institute for the Behavioral and Social Sciences (ARI) has developed a 
wide range of statistical models to test hypotheses generated in relation to an equally wide range 
of measurement and evaluation, situations. The powerful Randomized Block (RB)' design has 
traditionally been a preferred model for much psychological research. The RB design has the 
stringent requirement, however, that the sample population be strictly defined and stratified 
beforehand, a requirement more appropriate in a controlled laboratory environment than in many 
Army field situations. THis Technical Paper describes the development of an alternative statistical 
design which provides the advantages of the classic RB nr>ethod without its operational 
disadvantages, and which will be useful not only in the Individual Training and Performance 
Evaluation Technical Aj^a in which it was developed but in other areas of behavioral science 
research. 

The entire research is responsive to requirements of RDTE Project 2Q762717A745, Selection 
and Individual Training Research, FY 197& Work Program, and to special requirements of the 
Deputy Chief of Staff for Personnel. 
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A MODEL FOR USING QUALITATIVE VARIABLES AS COVARIATES IN THE 
ANALYSIS OF COVARIA.NCE 

BRIEF 



Requirement: 

To develop, as an alternative to the traditional Randomized Block (RB) two-way analysis of 
variance design, an equally efficient statistical model that will eliminate the RB's requirement for a 
priori stratification and sampling and, at the same time, retain the RB's ability to handle 
categorical concomitant variables. That is, to develop a statistical design with the advantages of the 
classic RB method without yts operational disadvantages. ^ 

Procedure: 

The statistical model selected for comparison and test was a modified analysis of covariance 
(ANCOVA) design that does not require previously selected stratified samples and? does 
incorporate the ability to handle categorical variables-the Categorical Analysis of Covariance 
(CANCOVA). The powers of fixed effects RB and CANCOVA using qualitative (categorical) 
concomitant variables were analytically and empirically compared. A Monte Carlo program 
simulated fixed effects analysis with two levels of treatment, one criterion variable, and a 
qualitative concomitant variable with three design types. The parameters which varied for eac?h 
design type were sample size, ratio- of numbers of row observations, eta, and magnitude of 
treatment effects. 

Findings: 

With relatively large samples, the RB and the CANCOVA designs yielded the same information 
in terms of component sums of squares. With small samples, the power relationship is a function of 
sample size, design type, and amount of heterogeneity. Empirically, no practical difference 'was 
found between the power of RB and CANCOVA when the samples are large. 

Utilization of Findings: 

Where the population cannot be well defined or stratified because of necessary administrative 
and/or physical constraints, pr'actical field experimentation can be undertaken with a precision 
comparable to the more expensive and laborious traditional RB design. The CANCOVA requires 
only that the subjects be sampled directly from the population and randomly assigned to the 
different experimental treatments. 
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A MODEL FOR USING QUALITATIVE VARIABLES AS COVARIATES IN THE 
ANALYSIS OF COVARI ANCE 



Experimental results are not only affected by treatments but by 
extraneous variation which often tends to mask the primary effects of 
experimental treatments. The effect of extraneous variation on experi- 
ri?ental results. is referred to as experimental error varianfie. In the 
behavioral sciences experimental error variance can be relatively large 
and influence the results of an experiment in such a manner that only - 
large treatment effects can be detected, and even these may be subject 
to uncertainty. By the careful design of experiments, it is possible 
to control sources of extraneous variation, reducing experimental error 
variance and increasing the precision of the experiment (precision 
refers to the power or ability of., a design to detect treatment effects). 

One way of controlling experimental error variance is by capitalizing 
on relationships between the experimental or dependent variables and 
external concomitant variables. External concomitant variables are 
variables which are measured prior to experimentation and "are not 
affected by the experimental treatment. For example, IQ cooild be used 
as an external concomitant variable to control for error variance due to 
difference in innate abilities. Two designs which employ external 
concomitant variables to control error variance are: (l^X- Randomized 
Block (RB), and (2) Analysis of Covariance (ANCOVA) . The RB experi- 
mentally controls error variance by using the external concomitant 
variable to stratify the samples assigned to the treatment categories 
into homogeneous groups called blocks, while the ANCOVA statistically^ 
controls error variance by using the linear regression of the dependent 
experimental variable on the external concomitant variable. 

Several researchers have compared the precision of RB and ANCOVA 
designs. Cochran^ found precision was directly related to the correlation 
of the concomitant and dependent variables. For correlations of less - 
than O.J, the use of RB or ANCOVA to increase the precision of the 
experiment was inconsequential, but as the correlation increases towards 
unity, sizeable increases in precision are obtained. For. large sample 
sizes, Cochran concluded that for experimental designs in which the 
relationship between the experimental and concomitant variables was lineat, 
the * precisions of the RB and ANCOVA are about the same. Cox^ found that 



^ Cochran, W. G. Analysis of covariance: Its nature and uses. 
Biometrika , 19^, 261-2&1. 

Cox, D. R. The use of a concomitant 'variable in selecting an experi- 
mental design. Biometrika , 19^, kkj I5O-I58. 

11 



RB provided greater preci'sion when the correlation between the variables 
was less than 0.0 cfnd ANCOVA provided greater precision only when the 
correlation was greater than 0.8, Cox's conclusion applies to designs 
with relatively small sample sizes and blocking levels assigne'ci on the 
basis of an underlying continuum/ i,e,, a rank ordering of the categories, 
of the blockingivariable. Feldt^ studied designs in which each cell had 
at least two observations. (Cox^s designs only had one observation per 
cell.) Feldt concluded that for correlations less than OA, RB resulted 
. in approximately equal or greater precision than ANCOVA; for correlations 
greater than 0.6 ANCOVA was "superior." . 

The findings of the above studies are only applicable to designs in 
which the concomitant variable is continuous, i.e., a variable which can 
take on any valae within a specified range. For example, weight is a 
continuous variable; it can take on any value within the possible range 
of values applicable to the object being weighed.' The' results of these 
studies do not apply to designs in which thfe concomitant variable is 
qualitative, i.e., a vai^iable which is categorical, in that it cate- 
gorizes or names; for example, different modes of instructiok, racial 
differences, differences in geographic origin, or social clat^ differences 
are all qualitative variables. When the concomitant variables are 
qualitative, it is traditional to use the RB technique. In the RB de^-^'.ga 
the population is stratified into homogenous groups .based on the cate- 
gories of the qualitative concomitant variable. Once the population has 
been stratified, random samples of subjects are selected from each strata 
and assigned to the different experimental treatments. In a laboratory 
setting, with a well-defined jDopulation, a priori stratification of and 
random selection of subjects from the entire population is easily ac- 
complisned. However, in field experimentation, where the population is 
not well defined and a priori stratification of the entire population 
is difficult due to administrative and physical restrictions, efiplojnient 
of RB designs can be difficult or impossible* In many ^situations the 
possible gain in precision is far outweighed by the necessary effort 
and expense of employing the RB design. 

\ ' . ^ 

The-vANCOVA offers a possible alternative to the RB design. ANCOVA 

does not require a priori stratification and sampling; the subjects are 

sampled directly from the total population and randomly assigned to 

the different experimental treatments. However, the traditional ANCOVA 

model was developed under the assumption that the concomitant variables 

were random and continuous; as such, the traditional ANCOVA is not 

applicable in situations where the concomitant variables are qualitative 

and therefore categorical. The purpose of this research is to develop a 

categorical A^JCOVA' (CANCOVA), i.e., an ANCOVA which :wi 11 allow tfie use 

of categorical variables, ""and to compare the precision of the CANCOVA 

with the traditional RB. 



^ 

Feldt, L. S. A comparison cf the precision oP three experimental 
designs employing a concomitant variable, Psychometrika , 1958, 23 » 
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REVIEW OF RELATED LITERATURE 



* Two of the tnost widely used techniques for increasing the precision 
of randomized experiments ■ are : ; (1) stratification or blocking of 
experimental samples (RB)^ and (2) analysis of covariance (ANCOVA). This 
review is limited to studie^s in which the precisions^ of RB and ANCOVA 
were compared, 

Cochran"* showed that^ for both RB and ANCOVA^ the gain in precision 
over completely randomizfed designs was a function of the size of the 
correlation coefficient o between the criterion variable Y and the 

concomitant variable X, If a is the experimental error variance when 

. y . 

no adjustment is employed^ then the adjustment by covariance reduces 
this variance to: 



where f is error degrees of freedom- The factor involving f is needed 
e e 

to take into ..account errors in the estimated regression coefficient for 
the bivariate sample. The adjustment by blocking reduces to: 

- (1 - )- (2) 

From equationa 1 and 2 it; is clear that for small valued of p (p < 0.5). 

the gain in precis io^^^'affordisd by using either RB or ANCOVA is likely to 
be inconsequential but as d increases towards unity, sizeable increases 

^n precision are obtained, Cochran concluded' that ft) r* experimental »->^r 
designs in which the relationship between X and Y is linear^ the precisions 
of the RB and ANCOVA are about the same (for large sample sizes). 

Cox ^ used two measures of imprecision as a basis for comparing the 
relative efficiency of RB and ANCOVA, The first^ true imprecision was 
based on the population value of the average error variance for the 
difference between two treatment means (adjusted for covariance when 
appropriate). The second.^ apparent imprecision was defined as the 



Cochran, 1957, op. cit, 
^ Cox,. 19'?7, op, cit. J 3 



product of the true imprecision and an adjustment factor^ based on error 
degrees of freedom. . The adjustment factor allows for a more meaningful 
compactson of the relative efficiency of two techniques which use the 
same number of observations but have different error degrees of freedom. 
Symbolically^ these two indices are: 




where S_ _ is the variance of the estimated difference between a pair 

of treatment means averaged over all pairs of treatment means f is the 

' e 

error decrees of freedom^ n is the number of observations per treatment 

group, a is the variance of Y within each treatment population o is 
y . r r , 

the linear correlation coefficient between X and. Y • and [2a^ (1 - n )/n1 

' y xy 

is the minimum variance of the difference between treatment means averaged 

over all pairs of treatment means. For any pair of designs based on the 

same total sample size comparison of the respective values of I will 

a 

indicate which of the two designs Is more efficient. 

Cox evaluated I and I for RB and ANCOVA using several ' combinations 
of tofal^ample size, Pj^* numbers of levels of treatment (t). Cox 

concluded that RB provided greater precision when o < 0,6 and ANCOVA 
provided greater precision only when > 0,8. V 
> 

It should be noted that designs used by Cox were not fixed effects. 
Cox assumed that the blocking levels were seletited randomly by ranking 
soibjects o-n.the blocking variable X subdividing the ranked subjects 
into t groups, and assigning one subject per block at random to each of 
the t levels of treatment, Thus^ the interaction of blocks and treat- 
ment provides an. appropriate error ter:m. On the other hand^ designs 
used in behavioral research are typically fixed-effects models with 
more than one observation per cell. 



^ Fisher, R. A/-. The design of experiments . London: Oliver and Boyd 
1949- . ^ 



Feldt' extended Cox^s study to designs in which the main and inter- 
action effects were fixed and each cell had at least two observations. 
All designs considered by Feldt were completely randomized having t 
levels of treatment with an equal number of observations. The satnples 
were assumed to have been drawn from t normally distributed populations 
with constant variance and a normally distributed concomitant linearly 
related to Y. The treatment population means and variances of the X 
variable and the within treatment correlation coefficients between X and 
Y were assumed to be equal across all treatment levels. 

Feldt used the same indices of imprecision as Cox; however^ they 
were modified to account for more than one observation per cell. Fe^dt 
concluded that' for < 0.4 RB resulted in approximately equal or 

greater precision than ANCOVA; for p^^ > 0-6 ANCOVA was "superior/* For 

re latively. high values of p^ and relatively small total sample size^ . 

the difference in precision in favor of ANCOVA was appreciable. This 

difference was attributed to the relatively small sample size not 

permitting the experimenter to employ* a sufficiently large number of 

blocking levels to exploit fully the valine of the concomitant variable 

in RB. Feldt noted that for p < 0.2 and small sample sizes neither 

^xy ^ 

ANCOVA nor RB yielded ^appreciably greater precision than a completely . 

randomized design. 

In all articles reviewed, the concomitant variable^ X^ was assumed - 
to be normally distributed. Further^ none of the articles was addressed 
to the ::ituatiQn in which the concomitant variable was qualitative. 
Feldt 's and Cox^s indices of imprecision assume that the control variable 
had an underlying continuous distribution and that each * category of the 
control variable had a definable variance, ^When X is qualitative the 
within-block variance of X is ^ero; therefore ^ these- indices cannot be 
used to compare the precision of RB and ANCOVA when the concomitant 
variable is qualitative. o 



METHODOLOGY 

The methodology discu&sio4, is divided into two sections, analytical 
and empirical. The analytical section i.nvplyes^a comparison of the 
power of RB and CANCOVA in .which the number'of obserVa^tions (^^) withiri 

each_ of the I categories .of the concomitant variable is constant. The 
empirical^ section compares the power of CANCOVA in which n^ is a random 

variable (RCANCOVA) with RB in which n_^.; is a constant; sample size is 

held constant for both designs. This comparison simulates the practical. 



Feldt , Iq^P , op. cit . 
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circumstances surrounding the probable implementation of these designs. 
The Introduction of n. as a random variable complicates the mathematical 
models making it only^f easible to compare the power of the RB and RCANCOVA 
designs by Monte Carlo methods. 



. Analytical 

Development of the analytical models is based on the General Linear 
Model (GLM). • Structural models for RB and CANCOVA were developed dnd 
appropriate parameter and design matrices were defined. Functional 
relationships were established between the RB and CANCOVA model components 
by comparing the parameter and design matrices of the respective models. 
Mathematical functions were developed for the RB and CANCOVA treatment 
effect F ratios. The power of the two models was compared by examining 
the relationship between the mathematical functions defining the respec- 
tive treatment effect F probability distributions. 

Empirical 

The Generation of Random Normal Samples . The generation of random 
normal samples for RB and RCANCOVA was accomplished by using RANDN.^ 
RANDN was called separately to generate sets of observations for each 
cell of a design. Each set of observations was generated from a popu- 
lation with a specified mean and standard deviation of 1. Cell means 
were computed from row mean values (row means' area function of the 
correlation coefficient^ eta, for the specific design being run) and^ 
treatment effect differences. A FORTRAN program was used to compute 
the values of row means for each of the possible design combinations 
run; program^documentation and a table" of .the row means used' for each 
value of eta are presented in Appendix A. ' 

.^ .Goodness-of-'Fit Tests . The randomness and goodness of fit to 
normality of the samples generated_by RANDN are dependent on the initia- 
tion number used in the generating process. Several starting numbers ^ 
were tested for the fit of the numbers they generated to a hypothetical 
nonnal distribution by means of a chi-square goodness-of-f it test. 
Documentation for the chi-square program is presented in Appendix B. 
In addition to these tests ^ data were generated based on J^OOO samples 
using each starting number^ for the^ 12 possible design combinations in 
which eta and treatment effect were both 0. The goodness of fit of the 
empirical; frequency of rejection of the null hypothesis of no treatment* 
effect to the expected, frequency of rejection under the central F ^ * 

distribution was determined for six nominal- alpha levels ,ol, -05, .lO, 
.25,. .50, -7^' using a 5"<^sgree-of-freedom chi-square goodness-of-f it test. 
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Empirical Power Comparisons , Only designs in which' I was equal to 
2, 39 and h were studied. Several combinations of parameters were used 
with each of these three design types ("design type" refers to the 
number of categories in the concomitant variable) . The parameters which 
were varied for each design type were : (1) sample size, (2)_ ratio of 
number of row observations, (5) eta, and (k) magnitude of treatment 
effect. Table 1 lists the values assigned to each parameter for each 
of the three design types. 



Table 1 



PARAMETERS FOR EACH OF THE THREE DESIGN TYPES 







Design Type 






Two-Category 


Three -Category 


Four^Category 


Parameter 


Variable 


Variable 


Variable 




Design 


Design 


Design 


Sample Size 


20 


56 


56 




80 


ikk 


224 


Karginal Ratio 








equal ri 


1:1 


1:1:1 


1:1:1:1 


unequal n 




k:l:.l 


4:1:1:1 


Eta 


0.0 


0.0 


0.0 




0.5 


0.5 


-0.5 




t).9 


0.9 


0.9 


Magnitude of 


0.0 


0.0 


O.'O 


Treatment Effect 


0.2 


0.2 


0.2 




,0.5 


0..5 


0.5 



Two sets of 5j 000 sample Sata generations were run for each of the 
k8 design combinations in which the values for eta and the treatment 
effect were non-zero. One set of data generations was analyzed by the 
RB subroutine and the other by the RCANCOVA subroutine of the Monte 
Carlo program (Appendix C) . For each run, empirical F ratios for 
treatment, interaction (RB), and heterogeneity of regression (RCANCOVA) 
were computed along with tfieir respective probabilif ies under the 
null hypotheses of no treatment effect, no inderac ticJfn effect, and 
homogeneity of regression. Each' of the computed probabilities for the 
treatment effect F ratios was compared to the 6 nominal alpha levels, 
and rejection rates were tabulated for each alpha level. For each 
sample generated, the computed probability *for interaction and 
heterogeneity of regression F ratios was compared to the .01 and .05 
alpha levels; those samples with interaction or heterogeneity of 



regression significant at the .01 and .05 levels were tabulated as sub- 
groups. The empirical re jection rate for the treatment effect F ratios 
in. each sufegroup was tabulated in the same manner as described for the 
total set of samples. 

The empirical power of a given design combination for a specified 
alpha level is equal to the proportion of times the null hypothesis of 
no treatment effect is rejected. The empirical powers of RB and RCANCOVA 
were compared a}: each' of the six nominal alpha levels. The statistical 
•significance of th'e^^comparisons was determined by using either of two 
statistics; the first is a z statistic defined by .Walker and Lev:^ 



2 = 




^Npq/NiNg 



where N-j^.- is the number of cases in population 1 for which the observed 
proportion of rejection is p^ ^ Ng is the number of cases in population . 
2 for which the observed proportion of rejections is pg ^ p = p^^ + pg ^ 
q = 1 - p^ and N = + Ng ; the second is a z' statistic defined by 
Haight 

y - X 
z ' = - 

vy + X 

where y is the frequency of rejections" in population 1 and x is the 
frequency of rejections in population 2. The z statistic is approximately 
a random variable with a normal distribution and the z' statistic is 
approximately a random variable with a Poisson distribution. Since the^^" 
binomial test becomes skewed and the normal approximation is inaccurate 
for proportions close to 0 or 1^ the^z statistic was used'for comparisons 
in which the population proportions were close to .5^ and the z'. statistic 
was used for -comparisQns in which the population proportions were close 
to 1 or 0. 



.RESULTS 

Analytical 

\, The followifig ..invent ions- were used in developing the analytical 
arguments : \^ 
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^ Walker^ H. M. , and Lev^ J. Statistical inference . New York: Holt^ 
- Rinehart and Winston, 1953 • 

^° Haight, F. A. Handbook of the Poisson distribution . New York: 
Wiley, 1967. ^ . > . . , 



1. Superscripts designate the model or design type: f = fall model 
r = reduced model, c = CANCOVA, and b = RB. . . ' 

2. Subscripts index a variable within a specific design: 1 - 
category of control variable, j = level of the treatment dimension, and 
k = Individual within an ij combination. » . 

5- Matrix notation is condensed by writing a column vector as its 
transpose row vector; for matrices involving repetition of elements, 
the following column vector notation is used: An is a column vector 

of n^ A's; e.g.^ 1^ is a column vector of 9 I's. 

The RB model is a two-way analysis of variance design in which ihe 
levels of the blocking variable correspond to the categories of the 
qualitative control variable. In the CANCOVA model the categories of 
the qualitative control variable are translated into sets of dichotomous 

covariables.ii If (m = 1, 2, I-l) is the m covariable score, in 
th 

the i (i = 1, 2, . . . , I) category of the qualitative control variable, 

then the value of for m = i is and the val-ue of X for m i is g; 

the values of d ^ g are arbitrary^ e . g . , d = 1 and g = o , or d = -1 and 
g = 1^ etc. Table 2 schematically represents the covariable allocation 
or blocking- strategy for a design in which the qualitative control 
variable has I .categories , 

ANOVA designs can be represented as special form^ of the general 
linear model (GLM).? The matrix form f or ^ sample of n scores based 
on a model with *p < n parameters is 

c* 

Y = X e + e , 

where Y is an ^(nxl) vector of random observations, is an (nxp) design 
matrix of known quantities, 9^ is a (pxl) vector of unknown parameters, 
and e is an (nxl) vector of . unobserved random errors, normally distrib- 
uted with E(e) = 0 and E(e e'j = la^ , where I is the'(nxn) identity matrix 
and a is the variance. Y and X represent observable d^ta^ whereas 0. and 
e are unknown. The least squares estimator of 9 is defined such tha"t 



e 'e == minimum. 
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Suits, D. B. Use of dummy variables in'regression equations. Journal 
of the American Statistical Association ^ uecember 1962, 548-551: 
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Dayton, C. M. An introduction to the general linear model . University 
of Maryland, Department of Measurement and Statistics Monograph 
College Park, Maryland, I969. ' ^ 
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Table 2 



SCHEMATIC OF RB (1x2) OR CANCOVA (1x2 WITH I-l COVARIABLES) 



Dichotomous Dummy 
.Covariables 

(X ) Treatment 
m 



... X^__^ Blocks 



1 0 ... 0 Y-, -, Y-, „ 

lln^ 12n^ 



0 


0 


... 1 




Y 

1-1,11. 


Y 

I-l, 21 








• • • • 


I-l 






^I-l" 


0 


0 


.... 1 






^I-l,2n^_^ 




•0 


0 


. . . 0 




Y 

ni 


■^121 














■ 




0 


0 


. . . 0 






Y 












^ ' 1 * 


^•2- 
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The valu<- of k which minimizes e^'e is found by Che solution of 



= 0 



whpre t = 1^ 2, ...^ p. The solution results in a system of normal 
fquations: 



X x'e = x'y 



If X is nonsingular, then (x'x) ^ exists and there is a unique solution 

_e = (x'x)'^^x'y 

The model for a score in the RB analysis is 

Y.. =.^^^^+«^/^U.(f^)+v(f') + e(^;') 
nk J i ^ ij 1 jk 

where the superscript (fb) denotes full model for RB^ [i^^^^ is an 

additive constant or grand mean^ ^ ( j = , 2 , . . j) is the effect 

of being in the j^^ level of the treatment dimension, d[^^ (i = 1, 2,^ 

th ^ ( fh") ^ 

I) is the effect of being in the i blocking level^ y), is the 

th ^ ( fh) 

interaction effect of being in the ij cell of the design and ^ 

th 

IS the error effect associated with the k observation in the 
. .th. ^ ' 

tj cell of the design. If the parameter and design matrices are 
tie fined using this model, the design matrix will be singular with 
column rank JI, Since the smaller order of this^ des ign matrix is (J+1) 
(^+1)^ in order fco remove, the singularity a total of (j+I+1) restrictions 
is needed. These restrictions can be generated from three classes of 
linear restrictions: 

J 

S cc =0 ^* (3) 



z e = 0 (4) 

i=l ^ 



J 

r. Z y =0 • . (5) 

i=l j=l 
2/1 



Under these restrictions, appropriate parameter and design- matrices are: 

(fb)'^ r (fb) (fb) (fb) (fb) (fb) (fb) (fb) (fb) ■ 

e - ^ ,Yii ,Yi2 '■••Xl^I-1 



(fb) _ 



1 

"^11 


1 




1 

^11 




0 

"^11 






0 

■^11 




1 


0 

^11 




. 0 * 

"^11 


1 

"^21 


1 

"^21 




0 

"^21 




1 

"^21 






0 

"^21 

^x 




0 

"^21 

ax 


1 . 

"^21 

« X 


» 


• 0 

^21 


m 


• 




m* 




m 












■ 






• 


• 




m 




• 






• 




■ 


■ 






• 


• 




m 




• 






■ 




• 


■ 






1 

I-l, 


1 

1 i-i, 


1 


0 

i-i. 


1 


0 

— n 

i-i. 


1 




1 

i-i. 


1 


d 

"Tl , , 
1-1,1 


0 

"Tl 

I-l, 


1 


. 1 

"Tl , . 
1-1,1 


Sx 






0 

"^11 




0 

"^11 






0 

"^11 

r 




0 


d 

"^11 




. 0 

"^11 


1 

"^12 


-1 




1 

"^12 




0 

"^12 






"^12 




-1 

■^12 


0 

"^12 




. 0 

"^12 


1 

"^22 


-1 

22 




\2 




1- 

"^22 






0 

22 




0 

22 


-1 

22 




. 0 

22 






















■ 


■ 






■ 






m 
















• 






• 






















• 






1 


-1 

2 '^1-1. 


2 


0 


2 


0 


2 




1 


2 


0 

"^1-1,2 


0 


2 


. -1 

"^1-1,2 


1 

"^12 


-1 

"^12 




0 

"^12 




0 

■^12 






0 * 

■^12 




0 - 
"^12 






. 0- 
"^12 



To conserve space the design and parameter matrices illustrated through- 
out the rest of this paper are 'restricted to designs in which the 
treatment' dimension has two levels; this restriction does not preclude 
generalization of the results to designs with more than two levels- 0/ 
the treatment climension. 
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The model for the same 'score under the CANCOVA analysis using 
separate wiUiin-col 1 regression is 

J iJ ijk jk 

where (he) is a superscript denoting a CANCOVA design using separate 

within-cell regress ion /'ix^ ^ is an additive coustant a^^^"^ is the 

th ' ' J 

effect of being in the j level of the treatment dimension , b is the 

th 

regression coefficient due to the regression of the i covariable within 
, ,th th 
the J treatment level on the Y scores within the j treatment level ^ 

^-f 4V ^^^^ value of the ^i^^ covariable score for the k^^ observation 

within the-j treatment level and e\^^ ^ is* the random error for the 

th ' J 
k observation within ^tbe j level of ' treatment . The design matrix 



using this model will be singular with column rank JI. Since the 
smaller order of the design matrix is JI+1 to remove the singularity"! 
restriction is needed: 



j=i 



j 



0 



Under this restriction, appropriate parameter and design matrices are; 



(he) (he) , , - 



•''l-l''^12'^22'---'^-l,2 



1 ■ 

"^11 


1 

"^11 




0 


... 0 


m 


• 

• 


• 


1 

"^21 


1 

"^21 


0 

"^21 


1 


... 0 

• "^21 








"^21 








• 


• 
• 


• 

• 






• • • 


0 


■ • • 




1 


0 


0 

"^1-1,1 


• . . 1 










1 

"^11 


0 

"^11 


0 


• • • 0 

"^11 




• 


• 


1 

"^12 


-1 

"^12 




• 




1 

•^12 


0 

"^12 


... 0 

"^12 ■ 










22 




0 




0 

22 

• 


^22 ; 

• 


... 0 

"^22 

n m 


S-1.2 


'^1-1,2 






i 

r 

1 * 


0 

~"l-1.2 


0 . 

~"l-1.2 


... 1 

"^1-1,2 


1 

"I2 


-1 

"I2 




• 

• 
• 


• 


0 

"l2 


0 

"l2 


. . . 0 

"l2 



ERIC 



2B 

• 13 



Both X and X ^ ^ are nonsingular matrices with column rank JI. Since 

both are based on full-rank models, they account for the same amount of 

J oo( ft)) -^(hc) 14. 15 

the total variance and SS^ ^ = SS^ \ 

e e 

- - ^ ' 

An alternative to the (he) CANCOVA is a model in which the pooled 
within-cell regression coefficient is used instead of separate within- 
cell regression coefficients. The model for a score is 

Y., = n^''^ + a['''^ + Z b, X, + e 
jk ^ 1 ijk jk 

where (c) denotes the CANCOVA model in which the pooled within-cell 

regression coefficient is used^ M-^^^' is an additive constant^ a^,^^ is the 

th ^ 
effect of being in the j level of the treatment dimension^ b^ is the 

pooled within-cell regression coefficient due to the regression of the 

i^^ covariable on the Y scores, X , is the value of the i^^ covariable 

th ^-^ th 

score for the k individual in the j level, of the treatment and e 

th th • 

is the random error for the k observatit)n within the j treatment ' . 
level. The design matrix for this model will be singular with column 
rank J+I+1. Since the smaller order of this design matrix is J+I^ in 
order to remove the singularity^ 1 restriction is needed: 

t a, ^ 0 
j-i J 

Under this restriction the appropriate parameter and design matrices 
are : 



.(c)' , 



I-l- 



Scheffe, H. The analysis of variance . New York: John Wiley & Sons, 

Vrn. ' • ^ • 

Graybill, F. A. An introduction to linear statistical models . New 
York: McGraw-Hill, lOGl. Pp. lOb-145. 

■ 24 . ■ 
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.(c) 



■^11 




1 


■^11 


0 


""ll 




X 


n 
U 






"^21 

• 


"^21 

* 


""21 

« 


""21 

• • 


m 

1 


1 


0 


0 


. . 1 






""l-l,l 


~"l-l,l 


J, * 




n 

""ll. 


n 

"^11 


. • 0 


1 


-1 


'1 


0 


• . 0 


~"l2 




'""12 


— n ^ 
12 • 




_ J 

""22 


n 

"^22 . 


""22 


n 


m 

1 


"1 


• 

0 


0 


1 






I— X, /J 








0 

■^12 


0 

■^12 





11 

'21 



I-l.l 

'22 



1-1,2 
\2 



If the data are such that for the i 



th 



covariable b,, = b. , ._ for j = 1, 

then the (c) and the (hc)*CANCOVA models are identical and | 

th 

accouiit for the same amount of total variance; however if for the i 

covariable b.. ^ b. . ,^ for some value of j. then the column rank of X^^^^ 

will be greater than the column rajik of ^ and the difference in Ahe 
amount of total variance accounted fot by the two models will be reflected 
in the difference between their respective error sum of squares In 
general, >the difference between the error sum of squareiS for the two 
models is . 



SS 



e e h 



(6) 



where SS, is the sum of squares due to heterogeneity, 
n 

A similar approach can be used with the RB model to define the sum 



\ 

r 

o 



,(rb). 



of squares for interaction (SS^ ''"'') 

Y 0 



j , then the 

appropriate parameter and design matrices for this reduced riddel are: 



If . =0 for all i. 



>(rb) _ 



(rb) ^(rb) ^(rb) Jrb) 



,3- 
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Graybill, lOGl, op. cit, 



2r, 

15 - 



0 



Si 




1 

"^11 


0 


. 0 
"^11 


Si 

• 


« 


0 

"^21 


1 

"^21 


. 0 

~"21 

• 


• 


• 

1 


0 

■ Ti 

^1-1,1 


m m 
m 

0 


• « 
. 1 

"~n ^ _ _ 
I--l,l 




1 


0 


0 


. 0 


1 

"12 


-1 

~"12 


1 

~"12 


0 

-"12 


. 0 

~"12 




"-In 

"22 


0 

~"22 


1 

~"22 


. 0 
""22 



m m 



fa 



1„ "-in 0 ... 1 

"l-l,2 "l-l,2 "l-l,2 "l-l,2 "2-1,2 
^12 "l2 "l2 "l2 ~"l2 



The difference in column rank between the full and reduced RB models is 
(l-l). This difference in column rank is due to the elimination 
of the interaction component (y^^ = O) in the reduced model; therefore^ 

-the difference between the reduced and full models' error sum of squares 
is due to interaction 



ss^^^) - ss^f^) = SS^^^ » (7) 

e . e Y 



Functional relationships can be establishe'd between the sum of squares 
components for the RB- and CANCOVA models by examining the least squares 
estimators for their respective parameter vectors 

|(rb) __ ^-^(rb)yxb)j-l ^(rb)'^ 

Since Y is the same score vector for both models and X^^^ - X^^^^ the 
least sqaares estimators are equal: 
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-(c, 











y. . . 




A(rb) 
= t) ■ = 


(rb) 

JO 

^1 




Y 












• 
• 
• 




• 

• 
• 


V 


■ • * , 

• 
• 


b ° 
I-l 











From equation o it can be shown that 

,(<:) 



- Of 



ss 



(fb) 



a 



= s-s 



(he) 



q: 



ss 



(-b) ^ ss(c} ^ „(fb) ^ „(hc) 



SS 



ss; 



-1 (8) 



.-..(9) 
(.10) 



By substituting the appropriate equalities from equations 9 ^0 
into equations 6 and 7 it can be shown that 



and since X^^^ = X^"'^^ 



SS, = ss^^) 
h Y 



(rb) (c) 
SS^ ^ - SS^^ 
e e 



(11) 



(12) 



Table 5 lists all the - component sums of squares for each modeV along with 
their respective degrees of freedom. 

The difference in the power of the RB and CANCOVA analyses tq detect 
treatme*nt effects is dependent on both the amount of interactiofi (hetero- 
geneity) present in the data and the' difference in degrees of freedom 
associated with the mean square error tenn (MS^) under each mode|:. For 

the (fl?) RB with I levels of blocking and J levels of treatment: 

. . •• MS^^^) 



-n. .-JI 



(15)- 



i7 



Table 3 



SUM OF SQUARES AND DEGREES OF FREEDOM FOR • 
RB AND CANCOVA ANALYSES 



RB CANCOVA 



Sum of 


Degrees of 


Sum of 


Degrees of 


Squares 


Freedom 


Squares 


Freedom 


• SS 

a 




r 

SS 

a 


J-1 




V 

I-i 


bb^* (^regression; 




ss 


(j-i) (i-i) 


SS 

e 


n. .-J-I+1 


- SS 

e 


n..-Jl 

\ 

i 


e 


JI..-JI 

(j-'i) (i-i) 



Note. The SSg for CANCOVA can 6e partitioned into IS^S^*^^^ which is the error sum of squares using separate within-cell 
regression coefficients, and SS^^ which is the sum of squ^es for heterogenity of regression, ^ 



and for the (c) CANCOVA analysis with I-l covariables and J levels of 
treatment 

MS^^^ = \,\ . - (14) 

The F ratio for the treatment effeat iti the (fb) RB is 

e • e 

and the F ratio for treatment effect in tt^e (c) CANCOVA analysis is 

, . MS (n. .-J-lf+l)MS_, 

a = ^ ^ . (16) 

, . * e - e 
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For sufficiently large sample sizes ^ " 



where PfF ^ is the probability that F > F under the central F distri- 

but ion when the sum of squares for interact ton (heterogeneity) is equal 
to 0. When the sum of squares for interaction (heterogeneity) is not 
equal to 0, the following relationships hold: 

" ^ f^rM i^a^) "henF^=F^>l, (17) 



where F^ \s the F ratio for testing the null hypothesis of homogeneity. 

of regression aud^T^ is the F ratio for testing the null hypothesis of • 

Y ■ ' 

no interaction, 

I. 



a a 



Given that "F^.^^ > F^/^^ the proof of equations I7 and 18 is 

(n. .-JI)MS ,.(n..-J-I+l)MS^ 
.(fb) „(0 



SS^. ^ ss^ 



e e . h' 



■(n..-Jl)MS (n. .-J-I+1)MS 

e eh 



Dividing by MS^? expand ing^ and combining like terms: 

^\ (j-i) (i-i) 

. • (n..-JI) 

e 

dividing both sides by (J-1) (I-l) / (n..-Jl): 



(n. .-JI)SS 



h 



(J-1)(T-1)SS^^'^) 



erJc 



since 

(n..-Ji)sSj^ ns^ 
(j-i)(i-i)ss^hO lns[^^) 

' - 19 - . . 



(20) 



>1 "(21) 



•V 



'^i^^J] 5 -.'i l^a^] "hen = > L 



From equations I7 and 13 it is clear that when = Fy = 1 the power 
of the ,RB (Pw(RB)) is equal to the power of the CANCOVA (Pw(C)), when = 
F^ > 1, Pw(RB) > Pw(C), and when F^ = F^ < 1^ Pw(C) > Pw(RB). 

The above relationships" only apply when the sample size is sufficiently 
large to negate any degrees of freedom differences in the distribution 
of the treatment effect F ratios for the RB and CANCOVA analyses. When 
the sample s^ze is relatively small^ the" degrees ■ of freedom difference 
between the two techniques^ along with the level of heterogeneity 
(interaction) in the samples^ plays an important part in determining the 
relative power of the two techniques . ' 

The following arguments are used to define the functional relation- 
ship between the amount of heterogeneity (interaction) in small samples 
and the relative power of the RB and CANCOVA techniques. . If tbe critical 

values at a given alpha level are known for F^^^^ and F^^^ and SS^^^^ = 

(fb) ^ a ' . e 

SS^ ^ is defined as a constant equal to 1 then the value of the 

SS^ = §S^^^^ can be computed as a proportion Of SS^^^^ = SS^*^^; the 

value of this proportion when Pw(RB) = Pw(C) is defined as a pivot 

fhc) 

point value (PV) for the pow^f function. When the ratio SS, /SS^ ^ = 

( f b ) ( f b ) * n e . 

SS^ ^ / SS^ ^ > PV, Pw(RB) > Pw(C), but when this r^tio is less than 

PV, then Pw(Rg)'< Pw(C). ^ : 

Table ) lists the PV values for alpha levels of .01 and .05 for two-, 
three-, and four-category control variables with sample sizes varying 
from (J'1-2) to (j-l-B) observations per cell. The algebraic argument 
used to compute the listed PV values is: ^ 

Given that SS*^^^^ = SS^^^^ = 1, then ^ ' 
e e ^ 

k Vf^^^) - g - (22) 



and 



Oi 1 + SS^/n..-.T-1+l ' 

h 



MS ' . \ 
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Table 4 



PIVOT POINT 


VALUE FOR TtlE .01 AND 


.05 ALPHA LEVELS 








Alpha 




Desig^Type 


Total Sample Size 










.-01 PV 


.05 PV 




8 


.5899 


.4585 


Two-Category 


12 


.1994 


.1690 


Variable Design 


16 


.1139 


.1019 




.20 


.0791 


.0728 




24 


.0604 


.0566 


• 


12 


.6277 


.5013 


Three-Category 


18 


.2283 


.2040 


Var lab le Des ign 


24 


.1371 


.1270 




50 


.0977 


.0923 




36 


.0770 


.0729 




16 " 


. .6350 


.-5093 


Four-Category 


24 


. , .2377 


.2IB3 


Variable Design 


32 


.1382 


.1301 




40 


• 0997 


.0982 


9 


48 


.0791 


.0761 



where F^ and F^ ^ are the critical values, at a given ^alpha level 

of the treatment effect F ratios for the RB and CANCOVA designs re- 

S'pectively^ and n.. is the total sample size. MS can be coTirf3uted as a 

' • f f b ) ^ 

function of the known constants F^ . n. . J, and I using the relation- 

ship defined in Equation 22 : ■ 

Substitution of K for MS in Equation 25 results in 

(c) ^ K 

a l+SS^/n..-J-I+l ' 

n 



transposing gives 



a 



Since PV = SS, /SS^^""^ and SS^^''^ = SS^^""^ = 1, then PV = SS, ; therefore 
h e e e ^ h . ' 

Equation can be used to compute FV yalues for different designs.^ 

Table 4 only lists F\( yalues for designs with sample sizes up to a maximum 

of (j-T-6). Total satn]jle sizes greater than (J-I-6) are sufficiently 

large that the differences in the distributions of the treatment ef^fect F 

ratios for the RB and CANCOVA techniques are negligible, and the PV values 

can be computed directly from Equation 2o. - 



Empirical 

None of the 12 null design combinations (eta = 0 and treatment effect = 
0) produced significant lack of fit using a chi-square goodness-of-f it 
test. Table 5 contains summary data for the power comparisons between^ 
RB and CANCOVA for all 48 design combinations in which eta and the 
treatment effect were non-zetp. The empirical power for each analysis 
is expressed as the • proport ion of samples in which the null hypothesis 
of, no treatment effect was rejected. The empiripal powers for both the 
RB and RCANCOVA analyses are given for*the .Ql, .05, and .10 nominal 
alpha levels for each of the 48 design combinations. • The power corii- 
parisons are divided into three groupings: (l) comparisons for .the , 
total number of samples generated (5,000), (2) comparisons for those 
samples in which interaction and heterogeneity were significant at the 
;Dl level, and (3) comparisons for those samples in Vhich interaction 
and heterogeneity were ^'ignificant at the .0"; level. 



DISCUSSION 

Analytical Results - 

If there is no interaction or heterogeneity of regression^ the least- 
square estimators of parameters and error terms for RB and dANCOVA are 
identical. I*n situations where there is interaction or heterogeneity 
of regression^ the error sum of squares for CANCOVA can be partitioned 
into a sum of squares for heterogeneity which equals the sum of squares 
for interaction under RB analysis^ and' an error sum of squares due to 
the use of separate within-cell regression coefficients which equals 
the error sum of squares under RB analysis. Given the situation where 
sample size, is fixed and the nqtnber of observations per level of the 
control variable is a constant, RB and CANCOVA provide the same infor- 
mation. \ , 

Even though the two techniques can provide the same information in* 
terms of component sums of squares , the relative powers of the two 
techniques are not necessarily equal. The power relationship was shown 
to be a function of Sample size, design type^ and amount of heterogeneity 
( interac:tion) . For situations in which £he heterogeneity (interaction) 

-^is equal tu zero,..MS^^^ < MS^^^; therefore, P (^^^^^^ < P (^^^^^j making 
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Table 5 



EMPIRICAL POWER LEVELS FOR THE U8 DESIGN COMBINATIONS IN WHICH 
ETA AND THE TREATMENT EFFECT WERE NON-ZERO 



Two-Category Designs 
Sample Size 20 



unequal 
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Treatment 


Effect 






2 




5 




.2 


.5 




Eta- 










• .3 












Model Power Comparisons 




















RB/RC 








■» 












.01 


RB 
RC 


.0187 
.0193 


.0203 
.0117 


.0^3 
.0560 


.0617 
.0660 


.0113 
.0153 


.01^7 
.01^ 


. .0613 
.0613 


.0550- 
.0613 


notui-iicix 
Alpha 


.05 


RB 
RC 


.0707 
. .0723 


.0683 
..0637 


.1813 
.1693 


.1317 
..1750 


. ^ .0520^ 
. . .0675 


.0753 
.0693 


.1850 
.1703 


.16^ 
.1793 




.10 


RB 
RC 


.1233 
..13?7 ^ 


.1280 
.1213 


.2900 
.2790 


. 2810 
.2773 


. 1100^ 

.1280 


.1503 

.1313 


.2923^ 
.2690 


.2787 
.2860 




RBOl/RCOl 










- 










.01 ^ 


RB 
RC 


.0000 
.0000 . 


.0000 
.0000 


,1U29 
VO938 


.1795 
.0000 


.0000 
.oil- 55 


.0370 
.031$ 


.1250 
.0968 


.0690 
.orjih 


Nominal 
Alpha 




RB 
RC 


.1111 
.0833 


.0508 
.0291*- 


.U286 
.1875 


.3077 
.1071 


.03^5 
.1818 


.1111 
.0958 


.U688^ 
.1935 


.2069 
.28^ 




.10 


RB 
RC 


.2222 
.1667 


.1765 
.0588 


.U286 
.3^38 


A103 


.1379 
.2727 


.1U81 
.1563 


• .7188^ 


.2069 
.28^ 




RBO5/RCO5 




















.01 


RB 
RC • 


.0126 
.00611 


.0301 
.0126 


.0725 
.0361 


.08^ 




.0325 
.01U5 


.0621 
.Oil-05 


.1269 
.0970 . 


Nom''ln4l 
Alpha 


.05 


RB 
RC 


.1282 
.061^1 


.0977 

.OkhO 


.188!+ 
.1325 


.23^ 
.16^ 


.0809 
.0656 


.0976 
.0870 


-2795^ 
,1216 


.2687 
.2388 


\ 


>.10 


RB 
RC 


.1731 
.1^7^^ 


.1729 
.1132 


.275^ 
.2590 


.33^ 
.251U 


.1838 
.1311 


.1870 
.1522 


•^057^.* 
.1959 


.3060 
.2761 


Sample kize 8o ^ 




RB/RC 
















i 




.01 


RB 
RC 


.0380^ 
.0507 


.OU73 
.OU8O 


.3207 
.3^07 


.3^13 
.3390 


.0397 
.01^20 


.0503 
.Oi^U3 


.3307 
.3^53 


.3507 
.3^10 


Nominal 
Alpha 


.05 


RB 
RC 


.1183^ 
.i600' 


.IUU3 
.IUI7 


.5603^ 
.6083 


.6050 
'.5983 


.1397 
.13^7 


.1510 
.13^0^ 


.^87 
.5960. 


.5917 
.58^ 




.10 


RB 
RC 


.2103^^ 
.2510 


.2337 
.2210 


.6710^ 
.7220 


.7120 
.7080 


.2180 
.221^.7 


.2?^7 
,2213 


.6813 
.6920 


.7110 
-.7073 




RBOl/RCOl 




















.01 


RB 
RC 


.0789 
.1111 


.0000 

.03^+5 


.2903 
.U09I 


.5000 
.i^333 


.0870 
.0278 


.0800 
.1000 


.7500^^ 
.2069 


.1759 


Nominal 
Alpha 


.05 


RB 
RC 


.1^9 

.2222 


.071^^ 
.103^ 


.5806^ 
.7??7 


.6905 
.6667 


.1739 


.2000 
.UOOO 


1.00^0^ 

.5172 


.3769 

.U3^8 




.10 


RB 
RC 


.2368 

.2222 


.1U29 

.3U1U 


.677^'' • 
.818? 


.6905 
.7667 


.2778 


.3200 
.U5OO , 


l.OOOOj^ 
.73^1 


.7692 
.6087 




RBO5/RCO5 




















.01 


RB. 
RC 


.01; 26 
.061^3 


.O27U 
.0^4.32 


.2933 
.315^ 


^3893 
.3^31 


.0507 
.oit.05 


.0816 
.OiK96 


.3^62 
.U118 


.36lt.3 
.3660 


Nominal 
Alpha 


.05 


RB 
RC 


.1560 
.2071 


.IU38 
.li^8l 


.5333 

Mlh 


.6211.2 
:5693 


.130^ 
.11^86 • 


.2109 
.1631 


.5897 
.59^8 


.621U 
, .^52 




.10 


RB 
RC 


•.20^ 

,2mk 


.2808 

.31^07 


,6200 
.711^ 


.6913 
.6788 


.19^ 
.. .2027 


.2925 
.2199 


.5897^ 
.7190 


.72li4' 

.6667 



33 



23 



Table 5 - Continued 



Marginal Ratio 
^Treatment Effect 
Eta 



ThreerrCategory Designs 
Sample Size 5o 
equal 



. 5 



unequal 



Model Power Comparisons 
RB/RC 





.01 


RB 
RC 


.0103 
.0200 


.0177 
.01^^-7 


.IUU3 

1 %7 


,1210 

, iibo 


.0207 

. U i f u 


,0240 


.1197 

1 on X 
. IcAJ ^ 


.l48o! 
, 1310 


Nominal 
Alpha 


.05 


RB 
RC 


.0765 

.Ob65 


.07i^7 
.0765 


• 3^53 
.3277 


.2925 

.2980 


.0827 
.0790 


.0823 
.0887 


. 7-' f / 
.2955 


. ^±xu 
.5035 




.10 


RB 

RC 


.1520 
.1557 


.1527 
,li^07 


.i^500 


.4183 
.4117 


.1523 
.1493 


.1510 

.1487 ■ 


.4223 
.4153 


.4^77 

.4235 




RBOl/RCOl 




















.01 * 


RB 
RC 


.0667 
.0000 


.1176 
,0000 


.3000 
.1500 


.0833 
.0370 


.1154 
.0345 


.1^9* 
.0263 


.0690 
.1335 


.5000^ 
.1143 


Mom "f n 21 1 

Alpha 




RB 
RC 


onnn 

. d\JUU _^ 

.06if 5 


. -1-7^5. 
.0952 


• 5500 
.3500 


.2003 
.1852 


.1923 
.1724 


.1^9 
.1316 


,544 8 
. .3333 


.5000^^ 
.1714 




.10 


RB 
RC 


. '>)00 


.Ull- 
. 11^29 


.6500 

.i^OOO 


.2500 
.2593 ' 


.2692 
.2759 


.1^9 
.1842 


.3795 
.^333 


.5000 
.3429 




RB05/RC05 




















.01 


RB 
RC 


.0501 
.0136 


.0567 
.020U 


.li^37 . 


.1293 

,1020 


• .0556 
.0252 


,0460 
.0291 


,2418^ 
.1497 


,2690; 
.1525 


Nominal 
Alpha 


.05 


RB 
RC 


.1566^ 


.1277 
.0864 


.5^38 


.3265 

.25;7 


.1528 
.0881 


.1034 
.1047 


,6144^ 
.5293 


.3655 ■ 
.3113 




.10 


RB 

i RC 


.i5t'5 


.2if&2^ 
.1361" 


:jjoh 


.39^6 
.3197 


.2451 


.16^7 
.2035 


.7320^^ 
%4551 


.5172 
.4437 


Sample Size Ikk 




RB/RC 




















.01 


RB 
RC 


.08if7 
.Oc50 


.P860 
.0817 


,6hoo 
.66po 


.6373 

.6427 


.0847 
.0873 


.0793 
.0853 


•r.6550 
.6460 


.6277 
.63^ 


Nominal 
Alpha 


.05 


RB 
RC 


" .22i^7 


.2210 
.2237 


.SI; 33 
.8400 


.8347 
.B313 


.2250 
.2250 


.2120 
.2180 


.5563 
.8^7 


,8450 
.8423 




.10 


RB 
RC 


.5227 


^.3173 
.3267 


.9040 
.9077 


.9B57 
.9080 


.^243 
.^337 


.3193 
.3273 


.9140 
.9177 


.9123 
.9073 




RBOl/RCOl • 




















.01 


RB 
RC 


.0000 
.0000 ' 


.1667 
.0^-70 


.5455 
.6522 


.6800 
.7333 


.1379 
.1818- 


.0541 
.0750 


.5152 
.3415 


.5000 
.6585 


Nominal 
Alpha 


.05 


RB 

RC 

t. 


.055^'* 

.5600 


.^333 
.217^^ 


.9091 
.7609 


.7200 
.8667 


.2414 
.!^182 


^ .1892 
.2500 


.7^6 
.7075 


.7568 
.8^3 




.10 


RB 
RC 


.1111 
.5600 


.5000 
.30i^3 


.9545 


.8400 

.9333 


.2759 
.3656 


.2973 

0500 


.9697 
.8537 


.9757 
.9756 




RB05/RC05 




















.01 


RC . 


.0682 
.0511 


.0701 
.0612 


.6395 
.6187 


.7105 
.^954 


.1500^- 
.0687 


.0588 
.0963 


.6017 
.5633 


.6493 
.6434 


Nominal 
Alpha 


.05 


RB 
RC 


.2117 


.2739 

.2585 


.7957 


.8750 
.8675 


.7929 
.2977 


.1830 
.2444 


.8559 
.8608 


.8881 
.8601 




.10 


RB 
RC 


.3197, 
.5^31 


.3694 
.3355 


.9252 
.8812 


.9211 
.9470 


.3786 
.3740 


,2876 
.5556 


.9?37 
.9114 


.9776 
.9441 



Table 5 - Continued 
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Mareinal Ratio. 






Four-Category Designs 
Sample Size ^ 
' . equal 




unequal 




Treatment 


Effect 






.2 














Eta 












.9 




.9 


.3 


.9 


Modefl Power Comparisons 




















RB/RC 




















.01 


RB 
RC 


.0517 
.0297 


.0290 
.0303 


.2353 
' .2097 


.2193 . 
•.2107 


.0320 
.0323 


.0300 
.0280 


.2173 
.2220 


,d.di\^ ( 

,2^0 


Nominal 
Alpha 




RB 
RC 


. 1113 
.1070 


. 11^3 
.1130 


.i^.3B0 


.i^.i^.60 
.i^.290 


.1053, 


.1183^ 
.1020 




'.1.503 




.10 


RB 
RC 


.1863 
.1820 


. 19^"3 
.1B93 


.6020^ 
.^20 


.56U7 • 
.5503 


.1750 
.1780 ' 


.1917 

M817 


.^03 


.5900 
.^33 




RBOl/RCOl 




















.01 


RB 
RC 


.0500 
.0000 


.0667 
.0000 


.3200 
.3000 


.3636^ 

* 

.1250 


. 222^ 
.0000 


.1290 
.0303 


"•^231^ 
.iii^-3 


.28^ 


Nominal 
Alpha 


.05 


RB 
RC 


.2000 
.0625 


.1000 


.5200 
.5000 


.5^55 
.3000 


.0870 


.1290 
.1212 


.^69 
.i^OOO 


.5862 

'.1.286 




.10 


RB 
RC 


.2500 
.09 3t 


.2000 
.2000 


,6800 
.6000 


.7727 . V 
.i^.250 


.5185, 
.217^ 


.1613 
.2U2l^■ 


.6151* 


.7586^ 

.5000. 




RB^:>yRC05 




















.01 


RB 
RC 


.0268 
.0192 


.0526 
.0129 


.3116 
.2553 


.$727^ 
.1355 


.0671 
.0355 


..06^ 
.0311 


,21^63 
.1667 


. 2500 • 

.2171 


Nominal 
Alpha 


.05 


RB ' 

RC 


.13U2 
,1026 


.13^2 
.08-39 


.5072 

.i^.6io 


.5035 

.5€7l 


.2256 ' 
.1560 


.1168 
.07^^ 5 


.5075 
.i^058 


.3953 




.10 


RB 
RC 


.20ei 
.1923 


.2039 
.1&71 


.6522 
.6099 


.636^^ 
.1903 


.2988 
.ail.82 


.1606 
.1739 


.5896 
.5362 


.6i4.8iv^ 
.5116 



RB/RC 



Sample Size 2^ 





.01 


RB 
RC 


- a331^^ 

.1363 


"•^11.67 
. .11.20 


.671.3 
.871.0 


.8727 
.B717 


.1237 
.1230 


.1587 
.1387 


.8730 
,863^ 


.8693 
.8683 


Nominal 
Alpha 


.05 


RB • 
RC 


.3ifco 

^3323 


.33^0 
.3267 


.9637 
.95B3 


.9660 
.9630 


.3250 

.3180 


.3173 
.3313 


.961+0 
.9593 


.9633 
.961+0 




.10 


RB 
RC 


.1.1.13 
.^550 


.1.617 
.li^97 


.98^3 
.9790 


.981.0 
.9810 . 


A1.53 


.1.1.17 
.1^1.53 


.9823 
.9833 


.9837 
.9813 




= RBOl/RCOl 




















.01 


RB 
RC 


.1765 
.oi.17 


.0690 
.I05i^-' 


.8^1 
.8333 


.9355 ' 
.8788 


.2083 

.1538 


.1379 
.1563 


.7917 
.7727 


.731.1 
.8^1 


Nominal' 
Alpha 


.05 


RB 
RC 


.29U1 
.2083 


.2069 
.1379 


1.0000 
1.0000 


1.0000 

.9697 


.3333 
.51.62 


.21. ll. 
.2500 


.9583 
.95^5 


.9310 
.9286 




.10 


RB 
RC 


.3235 
.3333 


.5^1.8 
.5^1.8 


1.0000 
1.0000 


1.0000 ■ 

.9697. 


.^583 

.1.615 


.1.138 
.3750 


1.0000 

.95^5 


1.0000 
1.0000 




RB05/RC05 

Rf^ 


.11.60 
.0800 


.679 
.1678 


.B599 
.9091 


.9167 
.9006 


.1799 

.1211. 


.11.19 
.159^ 


.8767 

.81.11. 


..8553 
.8130 


Nominal 
Alpha 


.05 


RB 
RC 


.21.00 
.2560 


. 3212 
.3221 


.9682 
.9805. 


.981.8 
.9752 


.3237 
.3500 


.3378 
.3623 


.9521 
.9517 


.9623 
.9512 




.10 


RB 
RC 


.3333 
.36-80 


.1.380 
.1.698 


.9809' 
.9935 


.9921. 
.9876 


.1.676 

.1.286 


.1.928 


.9795 
.9862 


.9871+ 
.9837 



* Pgwar compii<rr«on algnlflc«ntlv diffarvnt at .06 l»Smi. 
* * Powar companion tigniflcantiy different at .01 ItwK 



RC - RCANCOVA. 

RB01 - RB ftnalvtls in whiq)| intaraction i« s^fniflaint nt thm .01 

RB05 • RB nnaiyyis in which intaraction \t tlfniflecnt irt th« .05 I«v«i. 

RC01 - RCANCOVA a«»iy»in in which h»ttir09«n«ltv i« iif niflcant «t tha .01 iawwi. 

RCOS - RCANCOVA anaiyiis in which h«taTotanaitv is ^isn if leant at tha .06 i»v»l. 

/ « Comparad witti. 
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the CANCOVA technique uniformly more powerful under these conditions. 

This relationship does not hold, however, for conditions in wHich 

SS = SS is not equal to Q. A value equal to the ratio, SS, / SS^^^^ = 
n Y 'he 

PV, was defined for situations in which Pw(RB) = Pw(C). FV was shown to 

be a monotonic decreasing functian of sample siize. The smaller sample 

sizes.^ within any design type^ allow for the greatest amounts of hetero- 

greneity (^interaction) before Pw(RB) > Pw(C); in addition, the degrees of 

freedom difference between RB and CA'NCOVA techniques [df(c) > df(b)] 

provides a higher probability for rejection of the null hypothesis of ^ 

no treatment effect under CANCOVA, These combined factors provide 

CANCOVA with a power advantag-e when the sample size is relatively small 

and heterogeneity (interaction) is minimal. As the sample si^e increases^ 

'■Pt^ approaches 0 and at the same time the degrees of freedom advantage 

of CANCOVA over RB becomes negligible. Under these conditions the power 

advantage tends to shift in favor of RB. The relatively small FV of the 

larger sample size di^es not permit very high levels of heterogeneity 

(interaction) to be present before Pw(RB) becomes greater than Pw(C), 

Another aspect of the analytical results involves the implications 
regarding the relationship between power and the correlation between 
the concomitant variable and the* criterion variable. For RB and ANCOVA 
designs in which the concomitant variables are continuous^- the relative 
power of each design is dependent on the value of the linear correlation 
coefficient (p) between the concomitant and criterion variables. When^ 
p < 0.1, the RB technique tends to be more powerful; when p 0.6 ^ the • 
ANCOVA technique tends to be moi;e powerful. However, these results do 
not generalize to the case when the concomitant variable is not continuous, 
In RB, the qualitative condbraitant variable is used as a blocking variable 
and the amount of error reduction is equal ta the sum of squares for 
.blocking^ ^^3' CANCOVA, the qualitative concomitant variable is used 

to generate a set of dichotomous dummy Govariables and the amount of error 
reduction is equal to the sum of squares for the pooled within-cell 
regression SS^. It was shown that SS_ ="SS, ; since the amount of error 

variability explained by the correlation between the qualitative concomi- 
tant and criterion variables for both the ;RB and CANCOVA analyses is 
the same, the power difference between the RB or CANCOVA techniques is 
not a function of the value of eta. 



Empirical Results 

Goodness-of-f it . Using chi-square goodness-of-.f it tests, it was 
shown that all 12 null design <?ombinataons produced chi-square statistics 
which fell within 9*5^ confidence intervals for chance occurrences. These 
sample runs were used to establish the adequacy of the random number 
generators and to empirically validate the computational subroutines 
within the Monte Carlo prpgram. 
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Cox, 1957, op. cit.; Feldt, I958, op. cit. 



Empirical Fower Comparisons , The empirical data showed no inter- 
pretable differences^ either in magnitude and direction, ^between the 
power of the RB and RCANCOVA. At nominal alpha levels of. .01, .05, and 
.10 the number of significantly different power comparisons .for the 
two-, three-, and four-category design. types fell within 95^ confidence 
intervals for chance occurrences. Empirically there is no difference 
in power between the two techniques; however, caution must be exercised 
in interpreting these results to an applied situation. RCANCOVA was 
designed as., a post-hoc CANCOVA in which the a priori fixed number of 
observations per category of the concomitant variable (n^. ) ^^^Q'^^s ^ 

post, hoc random variable. Since CANCOVA. and RB were shown to be 
analytically equivalent, RCANCOVA could be viewed as a post hoc blocking 
technique; however, in^ actual practice post hoc blocking is a technique 
in vj^ich the a priori fixed number of observations per cell gf the 

design (n,,) becomes a post hoc random variable. Since n.. t n.."^ the 

ij 1 

RCANCOVA is not equivalent to a post hoc blocking technique; further 
investigation using more realistic post.hoc models is necessary before 
inferences can. be drawn regarding the relative power of RB and post 
hoc RB designs. 



SUMMARY 

The powers of fixed-effects randomized block (RB) and analysis of 
covariance (CANCOVA). using qualitative concomitant variables were 
analytically and empirically compared. Analytical comparisons were made 
jf the powers of RB and CANCOVA in which the number of observations (n.) 
within each of the I categories of the concomitant variable was a 
constant. Empirical comparisons were made of the power of CANCOVA in 
which n. 'was a random variable (RCANCOVA) with RB in which n_^ was a 

constant. A Monte Carlo program simulated fixed-effects analyses with 
two levels 6f treatment^ one criterion variable^ and a qualitative 
conccjrmitant variable with I categories. Three design types in which 
I was equal to 2, 5^ and 4 were studied. The parameters varied for 
eacli design type were-: (1) total. sample size (n..) (1=2^ n.'.=20, 80; 
1 = 5^ n..=5^:, 144; 1=4 n. . =^6 , 224), (2) ratio of number of row 
observations (l-2, 1:1 1:1; 1=^3, lUrl, 4:1:1; 1=4, 1:1:1:1, 4:1:1:1), 
(5)" eta (O.O, 0.3. 0.'9)j and (4) magnitude of treatment vef feet (O-O^ 
0.2. O.^y). 

Analytically ^ the RB and CANCOVA provided the same' information^ in 
terras orsfomponeut sums of squares. However, the power ire lationship was 
shown to be a function of sainple size^ design type, and amount of 
heterogeneity (interaction!^), present. Empirically no interpretable 
differences were found, either in magnitude and direction^ between the 
power of the RB and RCANCOVA for any of the design type and parameter 
combinations studied. 
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APPENDIX A 



DOCUMENTATION FOR PROGRAM WHICH COMPUTES ETA ^ ' 
FROM . MARGINAL RCW MEANS . ^ 

A single parameter card is used for each combination of marginal 
means tested. \ ' ' 



Col Information 

1 number of categories in the control variable (2^ 5^ or 4) 

2-4 value of marginal mean for first blocking level (F3.2) 

5-7 value of marginal mean for second blocking level (F3.2) 

'8-10 value of marginal mean for third blocking level (F3.2) 

ll-lj value of marginal mean, for fourth blocking level (F3.2) 



(card contains only as :many means as there are categories^ a three c 

gory control variable design has only three marginal mean values punched.) 

The program reads in the marginal mean values and computes the value 
" of eta based on the following relationship: 

lo* 1=1 - : 

T 

where I ^ number of rows [i^ - row mean^ = grand mean^ a? is the 

total variance and is set equal to pne. 

Output -consists of the values of the row means and the computed eta. 

if 

Table A-1 shows the row means for* each eta used in the Monte-Car Ig 
analyses . 

Table A-1 - . 

ROW MEAN VALUES FOR EACH DESIGN TYPE 







Control Variable 


Designs 








Value 


Two Category 


Three 


Category 




Four 


Category 




of 


Row 




Row 








Row 




Eta 


1 . 2 


* 1 


2 


• 5 • 


1 


2 


. 5 ■ ■ 


4 


.3 


l.OQ .1.60 


• 1.00 


1.30 


c 

1.70, 


t 

1.00 


1.20 


1.40 


■ i.8o 


.9 


1.00 2.8o 


• 1.00 


1.60 


3.75 


• .i.'^oo 


1.60 


2.20 


3.40 



!^ The row means in Table A-1 were used to compute the actual cell 
means for each cell in g specific design. The computed cell means were 
used byRANDN to generate the observations within that cell. For 
example^ if the value of eta was .'5 and the treatment effect was .2 for 
a two category design, then cell (11) ' s populat ion mean would be 0-90, 
cell (12Vs papulation mean would be l.lO, cell (21)'s population mean 
would be loa, and cell (22) »s population mean would be l.JQ^-/ 
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APPENDIX B 



DOCUMENTATION FOR THE CHI-SQUARE 
GOODNESS-OF-FIT TEST 

A single parameter card was used for each starting number to be 
tested. 

Col Inf oraat ion 

1-6 Soix -digit starting number read in under F format (F6.0) 

.The program read in the starting .number and called up the subroutine 
RANDN to genera ti^l^ 000 numbers^ with a mean of 0.0 and standard deviation 
of one. The program divides the empirical frequtacy distribution into 
, 16 intervals of ,k standard' deviations each. Observed frequencies are 
compared with the expected for each interval and chi squares for each 
interval as well as an overall chi square are computed. 

Output consists of the overall chi square^ observed frequencies for 
each o.f the 16 intervals ^nd chi squares for each interval. ^ 
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APPENDIX C 



DOCUMENTATION FOR MONTE CARLO PROGRAM 



The following parameter cards are used by the Monte Carlo program: 
/(l) Starting Nutnber Card 
Col- Information 



1-6 six difeit starting number for RANDN (f6.0) 

7-12 stx digit starting number for RANDU (F6.0) . 

15-16 iCKxx digit field for number of sample generations to be run 

for each design combination read in on the following parameter 
cards (ik) ' 

(2) ^ ' 

Col 



^1 0 stop* program 

1 read in parameters and run 

2 one digit F field for the value of eta, (Fl.l) ^ 

3-5 three digit F field for treatment effeo^t difference, (F^,2) 

6 one digit I field for number of cells in the RB analysis ^, 

' . 6, or 8 * ■ 

7-8 two digit I field defining number of observations to be " 

generated by RANDN in cell (11) 

9-11 ' three digit F field defining the value of the population mean 

under which the obserV^ations in cell (11) will be generated 
by RANDN,- (F3.2) 

12-13 two digit I field defining number of observations to be 

generated by RANDN in cell (21) 

li^-16 three digit F field defining the value of the population 

mean under which the observations in cell (21) will be 
generated by RANDN, (F3,2) ,^ 

Col 



i;epeat until par;amet'ers for all .cells involved in the particular design 
are punched. The maximum allowed is for a k x 2 design with 8 cells. 
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for f f '1'^' parameter cards; card (1) is read only once 

for each set of runs. There is no limit -to the number of type (2) cards 
wh.ch may be read using a single set of starting numbers, ir it is 
desirable to use different sets of starting numbers for each design 
combination, then the^ program must be terminated after each run 'and a 
new set of parameter cards, both (1) and (2), used to start a new run. 

RANDN is called up to generate data for each cell of the design. 
The data generated are analyzed by the RB subroutine. A new set of data 
IS generated by RANDN under subroutine RCANCOVA. This set ha^ the sa-^e 
number of observations as the RB analysis, but the number of observa^ons 
per contro variable category is determined by.a subroutine which r^^Zw 
assigns category classifications to each observation generated by RANDN i- f 

The probability of assigning a set of covariable scores corresponding to ' 
a given category of the concomitant variable is the ratio of the number 
of observations in that category to the total, number of- observations 
for the design. RCANCOVA produces a set of data in which the number of 
observations in each category of the control variable. is a random 
variable arid is representative of the population defined in the parameter 
card. Each observation within a category is assigned the proper set of 
covariable scores and the set, of .data generated under RCANCOVA .is -analyzed 
as If It were a one-way ANCOVA. , ' 

^ Output consists of a table containing the following: 
1. The design parameters 

a.. The dimensions of the design »• 
"' b. Eta 

. c. Treatment effect difference 



d. 



Cell sizes, treatment level sizes, blocking . level sizes, and 
the. total sample size 

^. Total sample data-- summary data for all 6,000 samples generated 

a. Frequency of rejection, proportion of rejection, z and 'z ' 
statistics at all six nominal alpha levels for the RB and 
RCANCOVA analyses 

b. A total, frequency count" for each analytical technique 

3. Interaction and heterogeneity at the .01 levels-data for all 
samples out of the 6, OOQ with interaction in the RB analysis 
significant at the .01 level and heterogeneity in the RCANCOVA 
analyses significant at the .01 level 



ai^ Same as 2a above 
b. Same ris 2b above 
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- 58 - 



Same as j above except for interaction and heterogeneity which 
was significant at the .05 level. 
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